Skip to content

Adopt the module-max SDK interface from dagger/dagger#13992 - #20

Merged
eunomie merged 9 commits into
dagger:mainfrom
eunomie:java-sdk-pr-13992-adoption-lead-2bf39fc4
Sep 8, 2026
Merged

Adopt the module-max SDK interface from dagger/dagger#13992#20
eunomie merged 9 commits into
dagger:mainfrom
eunomie:java-sdk-pr-13992-adoption-lead-2bf39fc4

Conversation

@eunomie

@eunomie eunomie commented Sep 4, 2026

Copy link
Copy Markdown
Member

Problem

dagger/dagger#13992 (branch sdk-ux-module-max, pinned here at 8fd9b22b5416f8dc7cb420ba37769adef6e874d2) changes how the engine drives an SDK module, with no compatibility adapter:

  • It removes CurrentModule.asSDK and ModuleSource.generateLocalDependencies. JavaSdk.modules selected the first and Mod.generateModule the second, so on that engine every call into this module failed.
  • It replaces the beta SDK-module interface (initModule, targetRuntime, @generate) with findClientRoot(ws) and generateScope(ws, isModule, name, clients). The SDK is recorded under [sdks.java] in dagger.toml, the engine sets Workspace.cwd to a persisted scope, and the SDK writes the module manifest itself through the engine's manifest builder.
  • Module dependencies are replaced by generated module clients.

This follows dagger/python-sdk#25 (merged), dagger/go-sdk#37 and dagger/dang-sdk#13, which adopt the same interface the same way.

Change

  • findClientRoot answers with the directory of the nearest pom.xml, relative to the workspace root, and null when there is none. Every Java module has one at its root, and the SDK vendored under a module carries none of its own, so the nearest hit is always the owning module — python-sdk has to lift a hit out of its vendored sdk/, Java does not. A project built with anything but Maven has no pom.xml, so it is correctly not a scope.
  • generateScope scaffolds a scope that has no config from the template, then generates it.
  • The manifest builder is no longer part of the engine. generateScope writes dagger-module.toml through github.com/dagger/sdk-helpers, pinned at 64645f1967d3dba6fce951dd61ae4acd8d9b0861. That module keeps the engine's rule that a non-builtin runtime is accepted on a manifest loaded from a config file, so this SDK can still name github.com/dagger/java-sdk/runtime.
  • The generate skip marker is gone entirely — the setting, Mod.skipGenerate, the marker files, and the fixtures that relied on them. generateScope always generates.
  • generateScope always writes dagger-module.toml, and migrates a pre-1.0 module: the contents of its dagger.json move into the new manifest and the dagger.json is removed, so the two cannot disagree. The runtime the module already named is preserved, so a module on the engine's builtin java runtime stays there.
  • The e2e module is no longer installed in dagger.toml. An uninstalled module is never reached by a released-engine dagger check, which is a stronger gate than the check.skip list it replaces; engine-e-2-e:sdk-contract-check runs dagger -m .dagger/modules/e2e check inside the playground.
  • In a module scope the client set becomes the module's dependency set. The manifest's dependencies are cleared structurally, with the builder's withoutLegacyRuntimeDependencies, not by name: withoutLegacyRuntimeDependency matches an unnamed dependency on its source, and reading the recorded names means resolving every one of them, so a single stale entry would fail generation instead of being dropped. A manifest that already records exactly the requested clients is left alone, decided by comparing the builder's rendering of both sides, so a hand-written one is never reformatted for nothing.
  • Standalone clients — a scope with clients and no module — are refused, as python-sdk refuses them: every generated binding lives under a module's sdk/ and is compiled by that module's pom.xml, so there is nowhere to put one.
  • Mod keeps its Maven machinery and gains generated, the workspace with the module's vendored SDK and entrypoint merged in. Its local-dependency staging is removed: the engine orders scope generation by dependency itself. generate, path and hasMarker go with the interface they served.
  • dagger.toml moves to [sdks.java] with one scope per fixture. [modules.sdk-sdk] goes with the contract it validates — it asserts initModule, the as-sdk marker and dagger module deps list, none of which survive. The dang-sdk as-sdk block goes too, with no replacement: dang-sdk has not adopted the new interface yet (Support new SDK interface (1.0.0-beta.12 and later) dang-sdk#13), and this repository's Dang modules have no generated files to produce.
  • The design note is hack/designs/2026-09-04-sdk-module-interface.md.

Verification

The released engine (v1.0.0-beta.11) cannot run this interface. Dang infers a whole program on each call, so on that engine every call into this module fails — measured, along with the fact that the module still loads, which is what makes uninstalling the e2e module sufficient. The e2e module is therefore uninstalled from dagger.toml and its checks are replayed inside a development engine instead.

  • engine-e-2-e:dev-sdk-check builds an engine from sdk-ux-module-max at 8fd9b22b inside Dagger. There, dagger sdk list reports java; dagger module init java --name sdk-smoke --path .dagger/modules/sdk-smoke writes dagger-module.toml naming github.com/dagger/java-sdk/runtime (and no dagger.json), pom.xml, the module class, sdk/src/main/java/io/dagger/client/Dagger.java and the generated entrypoint; and dagger -m … call container file --path /etc/alpine-release contents runs the module.
  • engine-e-2-e:sdk-contract-check replays the gated e-2-e checks inside that same engine, run as dagger -m .dagger/modules/e2e check: find-client-root-check, generate-scope-clients-check, generate-scope-init-check, generate-scope-migrate-check, nullable-return-check. generate-scope-migrate-check is new and covers the pre-1.0 dagger.json migration. This is coverage python-sdk#25 does not have.
  • generate-scope-clients-check covers both manifest formats — a client recorded in dagger-module.toml and in a pre-1.0 dagger.json, dropped again in both — asserts that a module with dependencies and no clients loses them, and generates a new module with a client to prove the client's type reaches the vendored bindings.
  • packager:unit-tests, packager:generate, templates:generate and dagger-dang-sdk:generate pass on the released engine.

Both the engine-dev dependency and the engine source name the same commit, so CI does not float with a branch that force-pushes; bump both, and dagger.lock with them.

Known limitations

  • On the released engine v1.0.0-beta.11, dagger call java-sdk find-client-root --ws . fails with field "withFile" not found in Dagger.WorkspacegenerateScope writes the manifest through Workspace.withFile and removes a migrated dagger.json through Workspace.withoutFile, and that engine has neither.
  • Existing modules lose their dependencies on the first generate. The client set is the whole dependency set, and the engine's config migration records only is-module and name on a scope — it never seeds clients from an existing dependency list. Each dependency must be re-registered as a client first. The README says so.
  • dagger module client add does not persist on sdk-ux-module-max at 8fd9b22b. It loses the workspace overlay on reload and silently writes nothing, on every SDK; the fault is in the CLI (internal/cmd/dagger/module_sdk.go), not in any SDK's generateScope. Client handling is therefore verified at the API level, calling generateScope from Dang and diffing the result — the same way python-sdk#25's checks avoid it.
  • Standalone module clients are not generated. A scope without a Java module is refused rather than silently skipped.
  • Two behaviours ship unchecked, deliberately: dependencySource's GIT_SOURCE arm needs a real remote no check here can produce hermetically, and no check runs a full generation of a pre-1.0 dagger.json module — its manifest branch is covered, the Maven half is not.
  • Manifest v2 (Load modules through manifest v2 entrypoints dagger#14038) and the unified-clients redesign (Modules have clients, not dependencies: generated clients for dependencies, self, and standalone use #17) are out of scope.
  • The engine branch force-pushes and renames as it goes: this PR already tracked detectScopefindClientRoot (now nullable) and the manifest builder's dependency verbs. Both engine-e2e pins and the docs name one commit, so a later move is a visible bump rather than silent drift.
  • A pre-existing race in mod.dang surfaced once the e2e checks began running in parallel: the shared Maven cache volume is seeded with a plain recursive copy, and two concurrent generations collide on it. The /root/.m2 mount is now LOCKED, so the mount serializes instead of the callers.

Design and implementation plan for replacing the beta SDK-module
interface (initModule, targetRuntime, the @generate hook, and
currentModule.asSDK) with detectScope and generateScope, as specified by
dagger/dagger#13992.

Signed-off-by: Yves Brissaud <yves@dagger.io>
dagger/dagger#13992 replaces the beta SDK-module interface with
detectScope and generateScope, and removes CurrentModule.asSDK,
ModuleSource.generateLocalDependencies, initModule, targetRuntime and
the @generate hook. It ships no compatibility adapter, so every public
function of this module selected something that no longer exists.

detectScope answers with the directory of the nearest pom.xml. Every
Java module has one at its root, and the SDK vendored under a module
carries none of its own, so the nearest hit is always the owning module
— python-sdk has to lift a hit out of its vendored sdk/, Java does not.

generateScope scaffolds a scope that has no config from the template and
writes its dagger-module.toml through the engine's manifest builder,
then generates it; a scope that already has a config is only generated.
A new module is generated even under a generate skip marker, which only
holds a module that already exists.

The builder has one runtime setter per builtin runtime, and this SDK
targets github.com/dagger/java-sdk/runtime, so the runtime is named by
loading a seed file: ModuleManifest.Validate accepts a non-builtin
runtime on a manifest loaded from a config file and rejects it on one
built from nothing.

In a module scope the client set becomes the module's dependency set.
The manifest's dependencies are cleared structurally, with the builder's
withoutDependencies, rather than by name: withoutDependency matches an
unnamed dependency on its source, and reading the recorded names means
resolving every one of them, so a single stale entry would fail
generation instead of being dropped. A manifest that already records
exactly the requested clients is left alone, decided by comparing the
builder's rendering of both sides, so a hand-written one is never
reformatted for nothing.

That cuts both ways — a module that records dependencies and has no
clients loses them — so the fixtures pin it, in both manifest formats.
Standalone clients, in a scope without a module, are refused, as
python-sdk refuses them: every generated binding lives under a module's
sdk/ and is compiled by that module's pom.xml, so there is nowhere to
put one.

Mod keeps its Maven machinery and gains `generated`, the workspace with
the module's vendored SDK and entrypoint merged in. Its local-dependency
staging goes: the engine orders scope generation by dependency itself.
`generate`, `path` and `hasMarker` go with the interface they served.

The SDK registers under [sdks.java] with one scope per fixture, and the
e2e checks move to the new interface. They call this module, so on the
released engine every one of them fails; [modules.e2e] check.skip covers
them.

sdk-sdk goes with the contract it validates: it asserts initModule, the
as-sdk marker and `dagger module deps list`, none of which survive.
The dang-sdk as-sdk block goes too, with no replacement: dang-sdk has
not adopted the new interface yet (dagger/dang-sdk#13), and this
repository's Dang modules have no generated files to produce.

Signed-off-by: Yves Brissaud <yves@dagger.io>
The released engine has none of dagger/dagger#13992, so it cannot run the
interface this SDK now implements: every call into the module fails with
`"moduleManifest" not found`. The checks that exercise the interface
therefore have nowhere to run on a released-engine `dagger check`.

engine-e2e builds an engine from the branch, at the commit engine-dev is
pinned to, and runs it as a playground with this checkout mounted
inside. dev-sdk-check initializes a Java module through the CLI and
calls it, which is the whole path: loading the SDK module, validating
its function signatures against the engine's contract, generateScope,
Maven, and the generated module serving its API. sdk-contract-check runs
the gated e-2-e checks in the same engine as `dagger call` invocations,
which `check.skip` does not filter.

Both the engine-dev dependency and the engine source name the same
commit, so CI does not float with a branch that force-pushes. Bumping
the branch means bumping both, and dagger.lock with them. The dependency
carries an explicit pin as well: without one, resolving it walks a local
path that does not exist and dagger-dang-sdk:generate fails.

Signed-off-by: Yves Brissaud <yves@dagger.io>
The engine no longer merges an SDK's init output with its own
bookkeeping: it records a scope in dagger.toml and asks the SDK to
produce the whole thing. So the commands change shape — `dagger module
install`, `dagger module init java --name … --path …`, `dagger generate`
— and `dagger call java-sdk init` / `generate --path` are gone with the
functions behind them.

Three things a reader now needs and could not find before: what makes a
directory a Java scope (a pom.xml), that a module's clients are its
whole dependency set, so an existing module's dependencies must be
re-registered as clients before the first generate, and which engine
each half of the check suite needs.

The "codegen flag" section goes. It documented `init` writing
codegen.automaticGitignore into a module's dagger.json; `init` is gone
and nothing here writes that key. What was worth keeping — the generated
files are committed, so the runtime skips codegen at load — the opening
paragraph already says.

Signed-off-by: Yves Brissaud <yves@dagger.io>
The branch moved from 78c241b6 to 7e6fc93c and renamed two things this
SDK selects, so this is not a re-pin.

detectScope is now findClientRoot, and its result is nullable: an SDK
reports "no client root here" with null, not with the empty string. The
engine reads the result through dagql.Nullable and treats an invalid one
as absent, so returning "" would have named a root at the workspace
root.

The manifest builder's dependency verbs are now explicit about whose
dependencies they are: withDependency, withoutDependency and
withoutDependencies became withLegacyRuntimeDependency,
withoutLegacyRuntimeDependency and withoutLegacyRuntimeDependencies,
matching the LegacyRuntimeDependencies field they write.

generateScope's signature and the [sdks.<name>.scopes] config shape are
unchanged.

dagger.lock is refreshed with `dagger update`: it carries the new
engine's build closure, and it also picks up dang-sdk's current head,
which had drifted past the recorded one.

Signed-off-by: Yves Brissaud <yves@dagger.io>
Removing it was wrong. The reasoning was that dagger/dagger#13992 drops
as-sdk, that dang-sdk has not adopted the replacement yet
(dagger/dang-sdk#13), and that a table the engine silently ignores is
worse than no table. The last step does not follow: the released engine
does not ignore it, dang-sdk reads it, and without a registration
dang-sdk's generator fails outright with "current module is not
installed as an SDK in this workspace" rather than reporting an empty
module set.

That failure was invisible here until dagger.lock was refreshed. The
recorded dang-sdk head predated the change, and a warm engine kept
serving the old resolution, so the check passed locally and in a fresh
clone while CI — which resolves the branch head — went red.

The module-max engine ignores the table, so keeping it costs nothing
there. It comes out when dang-sdk adopts the new interface and can be
registered under [sdks.dang] instead.

Signed-off-by: Yves Brissaud <yves@dagger.io>
The branch moved again, 7e6fc93c to 8fd9b22b, and took the module
manifest builder out of the engine with it. `moduleManifest` is no
longer a builtin: it lives in github.com/dagger/sdk-helpers, a Dang
module this SDK now depends on and reaches as
`sdkHelpers.moduleManifest(loadToml:)` / `(loadJson:)`. That module
declares engineVersion v0.21.9, so depending on it does not stop this
one loading on a released engine.

Three behaviours change with it, following dagger/python-sdk#26.

The generate skip marker is gone entirely — the setting, Mod.skipGenerate,
the marker files and the fixtures that relied on them. generateScope
always generates.

generateScope always writes dagger-module.toml from the builder, and
migrates a pre-1.0 module: the contents of its dagger.json move into the
new manifest and the dagger.json is removed, so the two cannot disagree.
The runtime the module already named is preserved, so a module on the
engine's builtin java runtime stays there rather than being moved onto
this repository's.

Path.relativeTo replaces the hand-rolled relative-path arithmetic. It is
on the released engine too, so it costs nothing to adopt.

The e2e module is no longer installed in dagger.toml. An uninstalled
module is never reached by a released-engine `dagger check`, which is a
stronger gate than the check.skip list it replaces, and
sdk-contract-check now runs `dagger -m .dagger/modules/e2e check` inside
the playground rather than naming each check.

Because generation is unconditional, a scope a check drives has to be a
module Maven can build. The config-only stub fixtures go, along with the
scope registrations that named them; the checks build their scopes from
the template instead and share one module name where they only need a
module, since the SDK installs its jars under a per-module Maven version.
generate-scope-migrate-check is new and covers the migration above.

sdk-helpers is pinned and locked. An unpinned dependency of this kind
already cost this branch a red CI run once: the resolution is cached
locally, so it keeps passing here long after it has moved upstream.

Signed-off-by: Yves Brissaud <yves@dagger.io>
Picks up "Omit the default module source from dagger-module.toml". The
builder's API is unchanged; only what it serializes moves, which the
manifest assertions in the e2e checks cover.

`dagger update` refreshes dagger.lock but leaves an explicit dependency
pin alone, so the pin in dagger-module.toml and dagger.json is bumped by
hand to the commit the lock now records. Letting the two disagree is how
a generate check goes red in CI while passing locally.

Signed-off-by: Yves Brissaud <yves@dagger.io>
`packager:generate` copied the whole io/dagger subtree out of the local
Maven repository. That repository is a cache volume shared with module
generation, which installs io/dagger/dagger-java-sdk/<module> and
io/dagger/dagger-java-annotation-processor/<module> into it under a
per-module version. So what landed under prebuilt/ depended on whether a
generation had run against the volume first, and the check reported
drift after a build that had succeeded.

Copy the two published artifacts and the metadata file by name instead.
The export no longer depends on what else is in the volume.

The mount is LOCKED here as well, matching the SDK's own codegen
containers: both write to that repository, and one volume takes one
sharing mode. Locking alone does not fix this — it orders concurrent
access, while the contamination is state left behind by an earlier
writer — but seeding the repository from prebuilt/ is a plain recursive
copy that two writers do race on.

The failure needed a full `dagger check` to reproduce: `dagger call
packager generate` on its own leaves the volume without the per-module
artifacts and reports no changes.

Signed-off-by: Yves Brissaud <yves@dagger.io>
@eunomie
eunomie marked this pull request as ready for review September 8, 2026 15:52
@eunomie
eunomie merged commit 179ab5c into dagger:main Sep 8, 2026
6 checks passed
@eunomie
eunomie deleted the java-sdk-pr-13992-adoption-lead-2bf39fc4 branch September 8, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant